1. Field of the Invention
The present invention relates to digital signal processing and, more particularly, to attenuation of aliasing artifacts in digital images.
2. Related Art
A conventional digital image may be represented as a two-dimensional array of pixels. Each pixel within the digital image may be uniquely identified by its column and row coordinates, which are typically numbered sequentially beginning with zero. For example, the upper-leftmost pixel of a digital image is typically specified by the coordinates (0,0) and, more generally, a pixel at column c and row r is specified by coordinates (c, r). Although a variety of conventional coordinate schemes exist, the coordinate scheme just described will be used herein for purposes of example.
Each pixel in a digital image has a value that specifies the color of the pixel. For example, in a monochrome (black-and-white) image, the value of each pixel may be either one of two possible values (such as zero and one), indicating whether the pixel is black or white. The values of pixels in a grayscale digital image are typically limited to a predetermined range of values corresponding to shades of gray ranging from pure black to pure white. For example, pixel values in a grayscale digital image may be limited to the range of 0–255, where zero represents black, 255 represents white, and intermediate values represent intermediate shades of gray.
In a color digital image, the value of each pixel corresponds to the pixel's color. The fact that the human eye has three different kinds of cones for sensing color enables us to represent all possible colors in a three-dimensional color space. A variety of three-dimensional color spaces can be employed to represent the same color. The axes of these various color spaces differ in the attributes of color that they represent. For example, one conventional class of color spaces is the red-green-blue (RGB) class of color spaces, in which the three axes correspond to red, green, and blue color components. Each three-dimensional coordinate within an RGB color space corresponds to a particular combination of red, green, and blue color components that uniquely specify a color within the color space. Another similar class of color spaces is the cyan-magenta-yellow class of color spaces, in which the three axes correspond to cyan, magenta, and yellow color components. Another class of color spaces represent the pixel's color in terms of luminance and chrominance components. The luminance component is related to the intensity or brightness of a pixel. This is the component that is captured on a black and white photograph or is displayed on a black and white TV. The two-dimensional space that is orthogonal to the luminance is spanned by the chrominance components. These components capture the color of the pixel. These two-dimensional chrominance spaces can be represented in either polar or Cartesian coordinates. When polar coordinates are used, the angle captures the hue of the color and the distance from the origin captures the saturation of the color. Examples of such color spaces are hue-saturation-brightness (HSB) and hue-lightness-saturation (HLS). Examples of standard color spaces in which the chrominance components are represented in a Cartesian coordinate system are LAB and YIQ.
In a color digital image represented according to a three-dimensional color space, such as an RGB color space, the value of each pixel includes three color components, each of which corresponds to a particular axis (e.g., the red, green, or blue axis) in the color space. Color component values are typically limited to integral amounts within a predetermined range, such as 0–255. For example, the red, green, and blue components of a pixel's color value may each be stored in a separate byte having a range of 0–255. The combination of red, green, and blue color component values for a particular pixel specify the coordinates of a point within the RGB space, and thereby specify the pixel's color. For example, a pixel having a red value of 127, a green value of 0, and a blue value of 255 specifies the color at coordinate (127, 0, 255) within the corresponding RGB color space. Typically, the value of a color component is proportional to the contribution of that color component to the color of the pixel. For example, a color having a large red component and small green and blue components will typically be rendered as predominantly red. Other color spaces may be encoded similarly. A variety of conventional techniques may be used for rendering a digital image on an output device such as a computer monitor or printer using appropriate colors represented according to a variety of color spaces.
Conventional digital cameras and other digital image acquisition devices may be used to acquire digital images from an image source, such as a printed page or a three-dimensional scene, and store the acquired image in a digital form as described above. For example, referring to FIG. 1, a conventional digital image acquisition system 100 includes a digital image capture system 102 that produces a captured digital image 104 from an original image 106. The original image 106 may be any source from which an image may be captured, such as a photograph, printed page, or real-life three-dimensional scene. The digital image capture system 102 (which may, for example, be a conventional digital camera), includes a conventional image acquisition device 108, such as a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) imager.
The image acquisition device 108 typically samples only a single color component (such as red, green, or blue) for each pixel in the captured digital image 104. This may be accomplished, for example, by superimposing a color filter pattern on the image acquisition device 108 so that only one color component (e.g., red, green, or blue) is sampled for each pixel in the captured digital image 104. As described in commonly owned U.S. Pat. No. 4,663,655 to Freeman, entitled “Method and Apparatus for Reconstructing Missing Color Samples,” a color recovery algorithm 110 may be provided to recover the color components that are not sampled by the image acquisition device 108. The output of the image acquisition device 108 is provided to the color recovery algorithm 110 to provide the missing color components at each pixel. For example, if only the red component of a particular pixel was captured by the image acquisition device 108, the color recovery algorithm 108 attempts to provide the missing blue and green components for the pixel.
Before describing the digital image acquisition system of FIG. 1 in more detail, the phenomenon of aliasing will be briefly described. Consider an analog signal whose highest-frequency component has a frequency of f0. As is well known to those of ordinary skill in the art, conversion of the analog signal into a digital signal by an analog-to-digital converter (ADC) typically involves sampling the analog signal at a frequency f1 and quantizing the resulting samples. According to the Nyquist Theorem, accurate reconstruction of the original analog signal from the digital signal requires that the sampling frequency f1 used by the ADC be greater than twice f0. This minimum frequency (2f0) required for accurate reconstruction of the original analog signal is commonly referred to as the Nyquist frequency or Nyquist rate.
If the original analog signal is sampled at a frequency lower than the Nyquist frequency, the reconstructed analog signal will contain signal components that were not present in the original analog signal. This phenomenon is referred to as aliasing, and the spurious signal components introduced by aliasing are referred to as aliasing artifacts.
Spurious low frequency sinusoidal signal components in the chrominance channel of the digital captured image 104 are one example of aliasing artifacts that may appear in the captured digital image 104. Such low-frequency sinusoidal components are typically produced by high frequencies in the original image 106 that are close to the Nyquist frequency and that are not sufficiently attenuated by an optical anti-aliasing filter 114 (described in more detail below). This kind of aliasing artifact typically manifests itself visually in the captured digital image 104 as spurious periodic color patterns that were not present in the original image 106.
A second kind of aliasing artifact may appear in the captured digital image 104 when a region of the original image 106 containing a sharp luminance boundary (such as black and white text) is sampled by the image acquisition device 108. Such sampling may introduce a phase difference between the red, green, and blue color components that appears as a spike or impulse in the chrominance channel of the captured digital image 104. This spike manifests itself visually in the captured digital image 104 as spurious color (also known as color fringes) in the vicinity of the region that contained the sharp luminance boundary in the original image 106.
Referring again to FIG. 1, the optical anti-aliasing filter 114 is typically provided between the original image 106 and the image acquisition device 108 in an attempt to attenuate aliasing artifacts such as those described above. The optical anti-aliasing filter 114 blurs the original image 106, such as by shifting the original image 106 by a small amount and superimposing the shifted image onto the original image 106 for delivery to the image acquisition device 108. An ideal anti-aliasing filter would eliminate from the original image 106 all spatial frequencies that are above the Nyquist frequency of the color filter pattern used by the image acquisition device 108. Actual anti-aliasing filters, however, typically only attenuate super-Nyquist signal components to varying degrees.
The visual effect of the optical anti-aliasing filter 114 is to blur the original image 106 for delivery to the image acquisition device 108. Conventional optical anti-aliasing filters are designed to zero the Nyquist frequency of the color filter pattern in the original image. However, these filters do not completely eliminate super-Nyquist frequencies because of the side-lobes that are present in their frequency response. Furthermore, the sub-Nyquist frequencies are also attenuated in this process, resulting in blurring in the captured image 104 that has aliasing artifacts. To reduce the hit in sharpness, some digital cameras employ optical anti-aliasing filters that cause less blur by expanding the filter's pass band beyond the Nyquist frequency. Although such filters produce less blur, they also cannot effectively attenuate super-Nyquist frequencies. When conventional color recovery algorithms are applied to anti-aliased images produced using such filters, the aliasing artifacts in the captured digital image 104 manifest themselves as color fringes and other visual imperfections.
When conventional optical anti-aliasing filters are used, there is a marked tradeoff between image sharpness and presence of aliasing artifacts. Stronger anti-aliasing filters reduce more aliasing artifacts at the expense of image sharpness. Although weaker anti-aliasing filters maintain more image sharpness, they attenuate fewer aliasing artifacts.
What is needed, therefore, is a system for attenuating aliasing artifacts without significantly affecting image sharpness.